Data Mining of Web Access Logs

Analysis of web visitors access patterns can lead to benefits in a wide range of areas such as decision support and website restructuring. Data mining techniques can be used to find access patterns hidden inside huge volumes of web access data. The goal of this thesis is to determine whether there are any such patterns in the web access data for the computer science website of RMIT university. In particular, this thesis investigates whether there are any differences in access patterns between: (1) Visitors from within Australia and visitors from outside Australia. (2) Visitors from within RMIT university and visitors from outside RMIT university. (3) Visitors from within RMIT university and visitors from outside RMIT university but within Australia. (4) Visitors from educational institutions other than RMIT university and visitors from non-educational institutions. The data mining techniques of classification, association rules, clustering and attribute selection were used with four different feature sets. The entire pattern discovery process was divided into three major steps: (1) Transaction identification and feature extraction (2) Discovery of the access patterns. (3) Analysis of the discovered patterns for their interestingness. Three major patterns were discovered: (1) Visitors from Australia generally visit the root page while visitors from outside Australia do not. The most likely reason for this is that visitors from outside Australia use search engines that direct them to specific pages. However, some (2) Visitors from outside Australia visit the root page and pages about post graduate programs (such as Master of Technology). This suggests that these visitors are mostly interested in post graduate studies. (3) Visitors from other educational institutions tend to visit pages related to staff contact information while other visitors tend to access career and industry related information. During the course of the investigation, it was found that there were a significant number of long transactions. The long transactions were analysed manually and it was found that visitors in a significant number of transactions access information about different programs offered and towards the end of their visit they look for the information brochure of one program. The significance of the patterns discovered during this thesis work suggests that data mining techniques with suitable feature sets can produce very interesting patterns.